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Objectives of the Talk 


• Assess the performance of NGAS Version-1 .5.03.00 CrIS/ATMS retrieval algorithm 
as delivered by LaRC, modified to include the MW and IR tuning coefficients and new 
CrIS noise model delivered by Degui Gu 

Percent acceptance 

RMS and mean differences of T(p) vs. ECMWF truth as a function of % yield 


• Compare performance of NGAS retrieval algorithm with an AIRS Science Team 
Version-6 like retrieval algorithm modified at SRT for CrIS/ATMS 

All experiments use NOAA CrIS/ATMS proxy data based on IASI/AMSU observations 


The next chart shows results we obtained using the NGAS retrieval system using the 
new CrIS noise level done both with and without NGAS IR and MW tuning coefficients 
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PRESSURE ( mb ) 


Percent of All Cases 
Accepted as a Function of Height 
Using NGAS Quality Control 


October 19, 2007 Global 
NGAS CrIS/ATMS Retrievals 

Layer Mean RMS Temperature (°C) 
Differences from ECMWF "Truth" 


Layer Mean BIAS Temperature (°C) 
Differences from ECMWF "Truth" 




NGAS Without Tuning 
NGAS Tuned 


Yield = 0.12% 
Yield = 0.14% 
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Results Using NGAS Retrieval System 

• With No tuning 

Results using no I R or MW tuning were very poor 

• Less than 1% of the retrievals were accepted using NGAS QC procedure 

• There were very large biases from ECMWF in retrieved quantities for the few 

accepted cases 

• There were also very large RMS differences from ECMWF for these cases 

This is not a surprise - tuning coefficients developed after launch are always needed 

• With IR and MW tuning 

Results using IR and MW tuning are better but still unacceptable for operational use 

• Less than 1% of the retrievals are still accepted using NGAS QC 

• Biases against ECMWF are much lower but still unacceptably large 

NGAS tuning coefficients appear to be sub-optimal 

• RMS differences from ECMWF are unacceptably large, especially near the surface 
These results do not mean much given that the % yield is so low 

We conducted a further evaluation of NGAS retrievals using cases in common with those 
accepted using the AIRS Version-6-like CrIS/ATMS retrieval algorithm 
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AIRS Science Team Version-6 T(p) QC Methodology 

All retrieved profiles T(p) have their own empirical error estimates 5T(p) 

Each profile has two pressures assigned to it, p best and p good , down to which the profile is 
considered of best quality and good quality where p best < p good < p surf 

There are two sets of thresholds A DA (p) and A CUM (p) used to determine p best and p good 
respectively 

p best is the pressure down to which 5T(p) < A DA (p) 

A da T(p) is a tight set of thresholds designed to give RMS errors of T(p) down to p best 

on the order of IK - these are called Data Assimilation thresholds 

Pgood ' s the pressure down to which 5T(p) < A CUM (p) where A CUM (p) are larger than A DA (p) 

Acum(P) is designed to maximize percent yield down to p good while providing RMS errors 
< 2K - these are called Climate thresholds 

AIRS Version-6 results for October 19, 2007 are shown on the next viewgraph 

• Demonstrates the concept of having 2 types of pressure dependent QC thresholds 

• Demonstrates the quality of results we should get using CrIS/ATMS data 

• Statistics include all cases down to p best and p good respectively 

Global percent yields at 500 mb are 60% and 96% using DA and CLIM thresholds 
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PRESSURE 


October 19, 2007 Global 
AIRS Science Team Version-6 Retrievals 


Percent of All Cases Accepted Layer Mean RMS Temperature (°K) Layer Mean BIAS Temperature (°K) 
Accepted as a Function of Height Differences from ECMWF Differences from ECMWF 

Using AIRS Version-6 Quality Control 




AIRS Version-6 Data Assimilation Quality Control 

AIRS Version-6 Climate Quality Control 
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SRT Version-6 Like CrIS/ATMS Retrievals 


Uses AIRS Version-6 retrieval code modified for use with CrIS/ATMS observations 
Uses ATMS RTA provided by Phil Rosenkranz in 2007 

Uses CrIS RTA Version-1 0A developed by Larrabee Strow and augmented by Eric 
Maddy for compatibility with SRT AIRS-like retrieval program 

Uses CrIS and ATMS channels analogous to those used with AIRS/AMSU 

No optimization has been done yet with regard to channels or other retrieval details 

SRT did update the following for CrIS/ATMS 

• Regression coefficients 

• IR and MW tuning coefficients 

• Error estimate coefficients 

• A DA (p) and A CL im(P) thresholds 

Next figures compare performance of SRT and NGAS CrIS/ATMS retrievals on two 
common sets of ensembles 

SRT cases accepted down to p best using DA thresholds 
SRT cases accepted down to p good using CLIM thresholds 
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PRESSURE 


October 19, 2007 Global 
CrIS/ATMS Retrievals 

Percent of SRT Cases 

Accepted as a Function of Height Layer Mean RMS Temperature (°C) Layer Mean BIAS Temperature (°C) 
Using AIRS Version-6-like Quality Control Differences from ECMWF "Truth" Differences from ECMWF "Truth" 




SRT Retrievals for Data Assimilation QC Ensemble 

SRT Retrievals for Climate QC Ensemble 

NGAS Retrievals for Data Assimilation QC Ensemble 

NGAS Retrievals for Climate QC Ensemble 
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PRESSURE 


Percent of SRT Cases 
Accepted as a Function of Height 
Using AIRS Version-6-like Quality Control 


October 19, 2007 Land 50°N to 50°S 
CrIS/ATMS Retrievals 

Layer Mean RMS Temperature (°C) 
Differences from ECMWF "Truth" 
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SRT Retrievals for Data Assimilation QC Ensemble 

SRT Retrievals for Climate QC Ensemble 

NGAS Retrievals for Data Assimilation QC Ensemble 

NGAS Retrievals for Climate QC Ensemble 
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PRESSURE 


Percent of SRT Cases 
Accepted as a Function of Height 
Using AIRS Version-6-like Quality Control 


October 19, 2007 Ocean 50°N to 5 
CrIS/ATMS Retrievals 

Layer Mean RMS Temperature (°C) 
Differences from ECMWF "Truth" 


Layer Mean BIAS Temperature (°C) 
Differences from ECMWF "Truth" 





SRT Retrievals for Data Assimilation QC Ensemble 

SRT Retrievals for Climate QC Ensemble 

NGAS Retrievals for Data Assimilation QC Ensemble 

NGAS Retrievals for Climate QC Ensemble 
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Comparison of NGAS and SRT Retrieval Performance on Common Ensembles 

By definition, % yields as a function of pressure increase going from Data Assimilation 
thresholds to Climate thresholds 

Data Assimilation thresholds includes an easier, but still very large, set of cases 
84% accepted globally at 500 mb 
Climate thresholds include a very extensive set of cases 
91% accepted globally at 500 mb 

Both SRT and NGAS results degrade as more difficult cases are included in the ensemble 

SRT CrIS/ATMS retrievals are considerably more accurate than NGAS retrievals for each 
common ensemble 

SRT RMS errors are much lower 
SRT retrievals are essentially unbiased 

This shows SRT derived tuning coefficients are performing well 
NGAS retrievals have large negative biases near the surface especially for climate 
ensemble 

This shows that NGAS cloud clearing methodology is not performing optimally 
for hard cloud cases 

Degradation in NGAS retrievals compared to SRT is greater over ocean than over land 

NGAS retrieval system needs significant improvement before it is ready for 
operational use 
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Improved SRT Methodology to Generate IR Tuning Coefficients A0 { 

Tuning coefficients A0, represent a term which is added to |© i _®c° m pj in the retrieval 
where 0j is the clear column brightness temperature for channel i and 0 i comp 
is a computed brightness temperature 

A0j is derived as the mean value of 0j -0? mth over an ensemble where 0? mth is 
©comp usjng ECMWF as “ tru th” 

Previous methodology used an ensemble comprised of clear ocean cases 
There are very few clear ocean cases, and some might not actually be clear 

New methodology used 0j for the 70% of the ocean cases in which p best = p surf 
See chart 10 

Charts 10 (ocean) and 9 (land) show SRT tuning coefficients perform well over ocean and 
land 

A0j is compatible with CrIS RTA Version-1 0A and available for everyone to use 
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